Publish Consistency and Producer-Consumer Locking: Efficiently Distributing Updates to Web Servers Using a File System

For low-latency access and scalability, Web site replication with a distributed file system has advantages over proxy caching. In particular, file systems encapsulate data consistency and recovery from failure behind a simple interface so that Web servers and Internet applications can operate as if they were on a single system. However, distributing content changes to many Web servers using a file system can heavily load the network between the servers, resulting in added latency. Poor performance arises in this environment because the Web-serving workload differs radically from the assumed file system workload. To address this shortcoming of file systems, we introduce the publish cache consistency model well suited to the Web-serving workload, which relaxes cache-consistency constraints slightly in order to enhance performance. We implement publish consistency in the producer-consumer locking protocol, in which the file system pushes content updates to Web servers. A comparison of this protocol against other file system protocols by simulation shows that producer-consumer locking removes almost all latency due to protocol overhead and significantly reduces network load. The work of this author was performed while a Visiting Scientist at the IBM Almaden Research Center.